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ABSTRACT 

This paper summarizes main findings from an investigation of familiarity and importance of 
assessment practices from teachers', counselors', and administrators' perspectives. The purpose of 
the study was to investigate to what extent participants are familiar with and deemed importance 
for their professional development a set of assessment practices. 

A survey with 74 practices, sampling seven areas of competencies, was assembled from The Code 
of Fair Testing Practices in Education, The Standards for Teacher Competence in Educational 
Assessment of Students, Career Counseling Competencies, and relevant assessment practices for 
Mexico. The survey inquired about degree of familiarity and degree of importance for professional 
development using a continuos score scale. 

The survey was administered to a sample of 200 participants from Southern Mexico. Results 
indicate participants' familiarity with a great number of the assessment practices. With regard to 
the importance of assessment practices for professional development, teachers' answers naturally 
clustered in three groups. Among the most important practices are the following: (1) skilled in 
choosing assessment methods for instructional decisions, (2) increasing reliability of test for grading, 
(3) skilled in communicating assessment results to students. Among the some what important 
knowledge and skills were found: (1) Examine specimen sets, disclosed tests, or samples of 
questions, (2) made assessment least measurement error, (3) acquiring knowledge about current 
ethical/legal issues with regard to the use of computer-assisted career guidance. The third group 
comprises the least relevant assessment practices: (1) Knowledge of state and federal status relating 
to client confidentiality, (2) describe the procedures that test takers or their parents/guardians may 
use to register complaints and to have problems resolved, and (3) interpreting grade equivalency 
score. 
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INTRODUCTION 



The school accountability system is undergoing a major revision in Mexico. Mexico's educational 
policy makers have endorsed benefits of having information on students' achievement from external 
evaluations. The school accountability process was developed for colleges and institutes first. One 
of the concrete efforts was the foundation of a National Center for Evaluation (CENEVAL, 1999) in 
1994 for developing tests for high stake decisions for high schools, colleges, and professional 
organizations. Recently policy makers' efforts are directed to developing an Institute for Evaluation 
to supply information regarding quality of education in Mexico's elementary and high school 
classrooms. 

In the near future a typical teacher in Mexico will spend a significant part of her time dealing with 
information derived from standardized assessment procedures. In the US, it has been found that as 
much as one third to one half of teachers' professional time is devoted to assessment practices 
(Stiggins and Conklin, 1992; Stiggins, 1999). Definitions of competencies to perform this job well 
are required in Mexico. 

Historically the assessment practices in Mexico have been connected to teacher-made instruments. 
Contrary to the U.S., in which the focus has been on standardized assessment practices (Stiggins, 
1999), Mexico has devoted most of the assessment training to increasing teachers' knowledge and 
skills for constructing instruments for classroom assessment purposes. Even though this practice 
need to stay, it is necessary to develop competencies on standardized testing, also. It is not 
uncommon finding teacher training programs with at least a measurement course focusing on item 
development, table of specifications, test assembling, and test administration from the perspective 
of teacher-made tests (Arce-Ferrer y Burgos, 1993). 

Assessment literacy is an important part of refining teacher-training programs in the US. Impara, 
Divine, Liverman, and Gay (1991) reported teachers' deficiencies on several topics related to the 
knowledge of testing principles such as using percentile bands and grade equivalent scores. Other 
researchers have documented knowledge and skills for counselors and educational administrators 
(Elmore, Ekston, and Diamond, 1993; Impara and Plake, 1993; Impara and Plake, 1995). 
Unfortunately, Mexico lacks studies targeting assessment literacy. 

Changes in Mexico's approach for bringing quality in education has activated a need for investigating 
assessment competencies required for being successful users of information derived from 
standardized achievement testing as well as other kinds of assessment practices. It is important to 
gather information regarding the current level of assessment literacy since an important part of the 
school accountability system is the use of assessment results from both standardized achievement 
testing and classroom testing. 



PURPOSE 
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The primary purpose of this paper is to define general competencies in assessment for pre-service 
and in-service teachers whose professional interests are in teaching, administration, or counseling 
areas. The main objectives are to: 

1 . Collect information about perceptions of familiarity on assessment practices such as 
choosing, developing, administering, scoring, interpreting, communicating, and 
legal/ethical issues. 

2. Gather information about perceptions of level of importance for professional 
development of assessment practices such as choosing, developing, administering, 
scoring, interpreting, communicating, and legal/ethical issues on professional practice. 



METHODOLOGY 



Survey construction 

Survey construction was guided by a set of seven standards similar to those utilized by Impara and 
Plake (1995). These seven standards are as follows: (1) Selecting assessment methods, (2) 
developing assessment methods, (3) administering, (4) scoring, (5) interpreting assessment results, 
(6) communicating assessment results, and (7) recognizing unethical practices. 

The Code of Fair Testing Practices in Education, The Standards for Teacher Competence in 
Educational Assessment of Students, and The Career Counseling Competencies were the source 
documents. Out of 74 items in the survey, 53 came from direct translation of competencies, 9 from 
cultural adaptation of competencies, and 12 were developed to target specific assessment practices 
in Mexico. Table 1 summarizes information about sources used for translated, adapted, and 
developed items. 



Insert Table 1 here 



Similar steps to those followed by Arce-Ferrer and Cisneros-Cohemour (2001) were implemented 
to achieve linguistic similarity and cultural equivalency in the translation of knowledge, skills, and 
abilities for each of three standards. 

1 . A bilingual professor with a doctorate in measurement and statistics from an American 
university translated the practices. Literal translation was conducted for each statement from the 
standards. 
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2. Another bilingual professor was asked to translate back to the source language. Similar 
instructions to achieve linguistic similarity as those given to the previous translator were given to 
this current sample of translators. 

3. Discrepancies in translation were resolved by agreement among the two independent translators. 

4. The two professors appraised cultural relevancy of behaviors for an initial set of 63 items. From 
this analysis, 9 items were modified to increase relevance and one item was dropped. The item 
dealing with interpreting stanines was removed since in only a few situations has it been used in 
Mexico. 

5. Twelve items were developed based on experience with assessment practices in Mexico. 



Sample 

The participants in the study were male and female pre-service and in-service teachers with 
professional interest in teaching, counseling, or administration from the southern region of Mexico. 
A total sample of 200 participants answered the survey. Approximately two-thirds were pre- 
service teachers. A little more than one-third of the sample has teaching as their main professional 
interest, and almost two-fifths of the total samples have counseling as their main professional 
interest. Almost a half of survey participants reported having taken between three to four courses 
in which assessment was the main content. Table 2 supplies additional information by professional 
interest about most frequently used assessment practices and decisions made from assessment 
instruments. 



Insert Table 2 here 



Pilot study 

The survey was piloted to assess participants' understanding of directions and questions. Results 
helped in identifying need for modifying question wording as well as identifying a suitable location 
for displaying survey scale. Regarding the latter, it was judged convenient having a scale for 
registering familiarity on the left side of the answer sheet and having another scale for reporting 
degree of importance on the right side. 



RESULTS 
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Main findings from the study are summarized in three sections. First, descriptive information 
about participants' familiarity is furnished. Second, survey results about participants' self-report of 
importance for professional practice are described. Finally, in the last portion, results from Impara 
and Plake (1995) are used to compare results from this study. 

Basic descriptive statistics was utilized to analyze survey responses. From a total of 200 
participants, 169 participants' answers were processed. Approximately 31 participants did not 
return the survey. The use of two follow-up letters increased participants' response rate by 3%. 

Familiarity with assessment practices 

Table 3 summarizes descriptive information about participants' familiarity with assessment 
practices by area and professional interest. 



Insert Table 3 here 



From Table 3, it can be observed that the lowest group means were for the assessment standard 
dealing with administration. It can be observed that teachers, administrators, and counselors 
reported having the lowest familiarity. For example, Table 3 depicts that approximately four-fifths 
of teachers were familiar with the set of test administration practices. Similar percentages were 
observed for administrators and counselors, also. 

On the other hand, the largest group means were observed for the standard connected to developing 
assessments. Most of the participants reported being familiar with developing assessment 
practices. 

In few occasions large differences were observed among members of each professional interest 
group regarding self-reported familiarity on each standard. Table 4 and Table 5 summarizes the 
three most familiar and the three less familiar skills and knowledge for each competency, 
respectively 



Insert Table 4 and Table 5 here 



Importance for professional practice 

Table 6 summarizes descriptive information for each of the three groups. It can be noticed that 
knowledge and skills related to assessment administration and assessment interpretation contains 
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the lowest mean percentage of importance for professional practice. On the other hand, the 
assessment practices with the largest means for the three groups were observed for the developing 
assessment area. On the average, the mean proportion of importance for the three groups was 
about 85%. 



Insert Table 6 here 



Even though, the above observations were not of statistical significance, they can be used 
qualitatively to guide specific search of important knowledge and skills in each of the seven 
competency areas. Table 7 presents relevant information about competencies for selecting 
assessment procedures. Similar tables, which are in the appendix, can be consulted for the other 
areas of competency. 



Insert Table 7 here 



The analysis at the survey item level revealed divergence among three groups regarding what is 
deemed important for professional practice. For example, in Table 7 it can be observed that teaching 
and counseling groups deemed as the most important skill choosing assessment methods whereas 
administrators valued most using them when complete assessment information is available. It can 
be observed that while for the administrators, the above behavior was the most important behavior 
(M = 91, SEM= 2.5), for the teaching and counseling groups were the second (M = 82, SEM=3.6) 
and the third (M = 84, SEM=3.0), respectively. 

Divergences among groups were observed for the less important region of the continuum, also. 

From Table 7, variability can be observed of what is deemed less important for professional 
practice for each of the three groups. For instance, in teachers' group opinion, reviewing the 
performance of test takers of different background was the least in importance and for counseling 
group it was evaluating the procedures used by test developers to avoid potentially insensitive 
content or language. A somewhat contradictory result emerges after identifying the less important 
knowledge and skill for the administration group. Table 7 indicates that examining specimen sets, 
disclosed tests or samples of questions is of no importance for their professional practice, even 
though a similar skill was judged as the most important before. 

To gain a better understanding of the results from this study, a comparison of measurement 
competencies of teachers, counselors and administrators in the U.S was conducted. Impara and 
Plake's (1995) results of a survey that examines the competencies of teachers, counselors, and 
administrators were considered as an external point to understand results from this study. It is 
relevant to acknowledge the qualitative value of the comparison between the two study findings. 
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While results from Impara and Plake's study arose from a test of measurement concepts, results 
from this study are from a self-report assessment of importance of measurement concepts for 
professional practice. Table 8 presents comparative results from the before mentioned studies. 
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Insert Table 8 here 



There are some similarities and differences between results from the two studies. As a single group, 
administrators, counselors, and teachers from the U.S performed higher in 1 1 out of the 20 
knowledge and skills than those in the study sample. U.S practitioners are more proficient than 
Mexico's sample on the following behaviors: 

• Selection of assessment method (basis). 

• Determining validity 

• Item construction —essay/performance 

• Interpreting teacher-made test score 

• Interpreting grade equivalency scores 

• Interpreting percentile band scores 

• Explain basis for grading 

• Using test for resource allocation 

• Displaying of grade —privacy 

• Test as only criterion for grade 

• Determining acceptable actions on standardized tests 

Another important information in Table 8 is the high discrepancies between the U.S. teachers and 
study sample teachers with regard to knowledge and importance of measurement concepts. There 
are some practices in which assessment knowledge of US teachers is larger than the degree of 
importance for professional practice in the study sample (Mexico). For example, while 94 percent 
of the U.S. sample answered correctly an item dealing with validity, the importance of this 
assessment concept was judged as being 75 percent relevant for professional practice. That is, 
while the concept of validity not only is fundamental but also teachers' competencies on it are high, 
its relevance for a sound professional practice is deemed not very important in Mexico's sample. 

There are other practices in which teachers from both countries are alike. For example, for the 
assessment practice dealing with using norms correctly, 77 percent of the US sample answered 
correctly on a test of measuring concepts and Mexico's sample judged 75 percent of relevance to 
professional work. 

Finally, there are some practices in which teachers from both countries shared similar responses. 
These are the practices regarding to low training and low perceived importance of training for 
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professional practice. For the standard dealing with interpreting assessment results, in two out of 
three behaviors, it was found both low achievement and low importance for professional practice. 
For example, for the behavior interpreting grade equivalency score. Table 3 indicates that around 
63% of the U.S. sample answered correctly a prompt measuring such a concept and Mexico's 
sample rated its importance low for their professional performance. 
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DISCUSSION OF RESULTS 



Mexico is moving into standardized achievement practices, and for maximizing such an investment 
of resources, it will be of interest to have a base line of its assessment literacy. Contrary to other 
countries, such as the U.S., use of standardized achievement tests, as part of the education quality 
assurance is new among most of school personnel. Most teachers, counselors, and administrators 
graduated from programs lacking training on standardized achievement testing (CENEVAL, 2000). 
An effect of a curriculum centered on teacher-made assessment arises from observing that 94 % the 
U.S. sampled teachers demonstrated competence in validity issues. The sample from Mexico 
acknowledged that validity knowledge and skills are important to their professional practice up to 
75%. Counselors seem not to be trained on assessment issues, either. While 91% of the U.S. 
sampled counselors demonstrated proficiency in meaning of measurement error, the importance 
assigned to such a practice in this study was of 68%. 

Finally, samples of administrators from the U.S. and Mexico performed similarly in most of the 
competency areas in the survey. However, for the competencies dealing with unethical assessment 
practices, the U.S. sample is more aware than Mexico's sample. On a test of measurement concepts, 
items dealing with privacy of information, using test as only criterion for grade, and taking 
acceptable actions on standardized tests obtained large p-values. However, same content was judged 
partially relevant in Mexico's sample. It is relevant saying that a similar scenario was observed for 
counselors and teachers, also. 

A second comment is directed to a need for college programs and professional organizations to 
move toward defining assessment competencies for a fair practice. We expect seeing a growing 
movement in some professional organizations in Mexico to define their assessment competencies. 
To catalyze such a development, it is important to be aware about potential benefits when having 
an initial set of knowledge and skills deemed valuable for assessment practices. This initial set of 
competencies might help during curriculum revision of teacher, counselor, and administrators 
programs, for preparing professional improvement courses, and perhaps creating a debate about 
whether developing or not state mandated teacher, counselor, and administrator certification 
programs. 

A third comment is directed to a need for defining an accountability system for supervising testing 
practices around Mexico. In spite of Mexico's desire to develop a fair testing system, there are no 
state and federal statutes to protect test users against testing malpractice. As a difference from the 
U.S. in which testing companies are accountable for unethical practice, there is no organization 
available in Mexico to defend test users against malpractice from testing companies. This situation 
is of concern when we take into account the lack of academic programs available to train 
measurement specialists in all Mexico. 

Finally, findings from this survey need to be seen only as initial results. The lack of a larger sample 
size reduces the generalization of results to all Mexico. There is a possibility of finding school 
personnel with different opinions to those from the sample. In addition, it is relevant to 
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acknowledge the confounding of training and importance. For Mexico's sample, training on some 
assessment concepts might be taking place and even though their perception of importance for 
professional practice is regarded low among school personnel such as teachers, counselors, and 
administrators. We acknowledge that the opposite, not being trained, but perceiving the practice 
important might be the most likely. Further research to assess the degree of assessment literacy 
will supply valuable information in addition to the one gathered on this self-reported study. 
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Table 1: 



Sources for survey items 



Standard 


Selecting 


Developing 


Administering 


Scoring 


Interpreting 


Comunic: 


Code of fair testing 


T=8 


T=2 


T=0 


T=0 


T=4 


T=0 


practices in education 


A=1 


A=0 


A=0 


A=0 


A=0 


A=0 


N=0 


N=0 


N=0 


N=0 


N=0 


N=0 


Standards for 


T=2 


T=3 


T=0 


T=4 


T=7 


T=1 


Teacher Competence 


A=2 


A=0 


A=0 


A=0 


A=1 


A=0 


in Educational 
Assessment of 
Students 


N=0 


N=0 


N=2 


N=3 


N=4 


N=3 


Career counseling 


T=6 


T=0 


T=1 


T=0 


T=1 


T=2 


competencies 


A=1 


A=0 


A=1 


A=0 


A=0 


A=0 


N=0 


N=0 


N=0 


N=0 


N=0 


N=0 


Totals 


T=16 


T=5 


T=1 


T=3 


T=12 


T=3 


A=4 


A=0 


A=1 


A=0 


A=1 


A=0 




N=0 


N=0 


N=2 


N=4 


N=4 


N=3 



T=Direct translation 
A=Linguistically and culturally adapted 
N=Nevvly developed for the study 
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Table 2: 

Assessment instruments and use of assessment results by professional interest 



Professional 

interest 



Assessment instruments 





N 


My own 
classroom 
test 

% 


Standarized 

test 

% 


Others 

classroom 

test 

% 


Oral 

test 

% 


Essay 

% 


Term 

papers 

% 


Portfolios 

% 


Expe 


Teaching 


62 


53.23 


17.74 


9.68 


20.97 


16.13 


11.29 


1.61 


2 


Administration 


29 


27.59 


34.48 


10.34 


3.45 


6.90 


6.90 


3.45 


e 


Counseling 


69 


26.09 


33.33 


8.70 


10.14 


5.80 


4.35 


1.45 


i 



Professional 

interest 



Uses of assessment results 







Grading 


Admission 


Diagnostic 


Placemen 


Formativ 


Scholar 


Counseling 


Voca 






% 


% 


% 


t 


e 


Ship 

% 


% 


orien 

( 




N 








% 


% 








Teaching 


62 


50.00 


11.29 


20.97 


4.84 


9.68 


0.00 


6.45 


4. 


Administration 


29 


31.03 


10.34 


10.34 


0.00 


6.90 


3.45 


3.45 


13 


Counseling 


69 


27.54 


8.70 


10.14 


2.90 


4.35 


2.90 


21.74 


17 



ERIC 



15 



Table 3: 

Descriptive information about participants' familiarity with assessment practices by area and 
professional interest 



Professional 


Selecting 


Developing 


Administering 


Scoring 


Interpreting < 


Interest 


% 


SE 


% 


SE 


% 


SE 


% 


SE 


% 


SE 


Teaching 


88.5 


(2.01) 


91.8 


(2.00) 


79.8 


(3.14) 


87.5 


(1.72) 


87.6 


(1.80) 


Administratio 

n 


87.6 


(3.64) 


92.4 


(2.70) 


83.0 


(4.98) 


90.4 


(2.52) 


87.6 


(3.09) 


Counseling 


92.0 


(1.66) 


87.2 


(2.47) 


80.9 


(3.10) 


88.4 


(2.39) 


86.5 


(2.09) 
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Table 4: 

Most familiar knowledge and skills for school administrators, counselors, and teachers 



Most Familiar 3 



Item description 





A 1 


C 2 


T 3 


Competency I: Selecting 








Knowledge about instruments and techniques to assess leisure. 


100.0 


95.6 


88.7 


Read the materials provided by test developers and avoid using tests for which unclear 
or incomplete information is provided. 


100.0 


97.1 


91.9 


Knowledge about instruments and techniques to assess personal characteristics. 


96.5 


100.0 


93.5 


Competency II: Developing 








Should be skilled in developing assessment methods. 


93.1 


88.4 


93.5 


Ascertain whether the test content and norm group(s) or comparison group(s) are 
appropriate. 


100.0 


98.5 


96.7 


Competency III: Administering 








Should be skilled in administering teacher-produced assessment methods. 


92.8 


83.8 


91.9 


Ability to administer, score and report findings from career assessment instruments. 


79.3 


88.4 


82.2 


Competency IV: Scoring 








Reliability of tests for grading. 


96.5 


95.6 


98.3 


Should be able to explain the rationale for choosing grading practices. 


85.7 


92.7 


95.1 


Explain basis for grade. 


93.1 


92.6 


95.1 


Competency V: Interpreting 








Should be able into consideration students characteristics as part of score interpretation 


100.0 


92.7 


93.5 


Should be skilled in using assessment results when making decisions 


93.1 


91.3 


98.3 


Obtain evidence to help show that the test is meeting its intended purpose(s). 


89.6 


89.8 


98.3 


Competency VI: Communicating 








Should be skilled in communicating assessment results to students. 


89.6 


92.7 


96.7 


Ability to write a thorough and substantiated report of assessment results. 


96.5 


94.2 


91.9 


Should be skilled in communicating assessment results to parents. 


89.6 


91.3 


98.3 


Competency VII: Ethical/legal Issues 








Knowledge about acceptable actions on standardized tests. 


93.1 


97.1 


88.5 


Display of grade-privacy. 


93.1 


98.5 


95.1 


Provide test takers the information they need to be familiar with the coverage of the test. 


93.1 


94.2 


86.9 



a On a self report of familiarity 

1 Percentage for administrator 

2 Percentage for counselor 

3 Percentage for teacher 
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Table 5: 

Less familiar knowledge and skills for school administrators, counselors, and teachers 



Item description 


Less Familiar 3 


' 


A 1 C 2 T 3 



Competency I: Selecting 



Examine specimen sets, disclosed tests or samples of questions. 


68.9 


88.4 


72.5 


Knowledge about instruments and techniques to assess works conditions. 


79.3 


88.4 


87.1 


Review the performance of test takers of different backgrounds when samples of 
sufficient size are available. 


79.3 


81.1 


75.8 


Competency II: Developing 








Item construction — essay/performance. 


89.6 


82.6 


88.7 


Made assessment least measurement error. 


89.6 


78.2 


90.3 


Competency III: Administering 








Should be skilled in administering externally-produced assessment methods. 


79.3 


79.7 


67.7 


Knowledge of and ability to effectively and appropriately use computer-assisted 
assessment measures and techniques. 


79.3 


72.4 


77.4 


Competency IV: Scoring 








Should be skilled in scoring externally-produced tests. 


79.3 


78.2 


61.2 


Weighting test scores to give grades. 


92.8 


85.5 


82.2 


Should be skilled in scoring teacher-produced tests. 


93.1 


85.2 


87.1 


Competency V: Interpreting 








Know basis for comparing schools' test scores. 


68.9 


72.0 


70.0 


Interpret percentile band scores. 


79.3 


79.4 


65.5 


Interpret Grade Equivalency score. 


68.9 


66.6 


65.5 


Competency VI: Communicating 








Explaining discrepancy between classroom and standard test scores. 


75.8 


77.9 


80.6 


Ability to assist client and others designated by the client to interpret data from 
assessment instruments 


89.6 


76.8 


83.8 


Competency VII: Ethical/legal Issues 








Knowledge of state and federal statutes relating to client confidentiality. 


65.5 


68.1 


60.6 


Describe the procedures that test takers or their parents/guardians may use to register 
complaints and have problems resolved. 


72.4 


68.1 


70.9 


Knowledge about current ethical/legal issues with regard to the use of computer-assisted 


79.3 


79.7 


73.7 



career guidance. 
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a On a self report of familiarity 

1 Percentage for administrator 

2 Percentage for counselor 

3 Percentage for teacher 



Table 6: 

Descriptive information of self-reported importance of assessment practices by area and 
professional interest 



Professional 

interest 


Selecting 


Developing 


Administering 


Scoring 


Interpreting < 




% 


SE 


% 


SE 


% 


SE 


% 


SE 


% 


SE 


Teaching 


83 


(1.5) 


85 


(1.6) 


80 


(1.8) 


83 


(1.5) 


79 


(1.5) 


Administratio 

n 


86 


(1-9) 


87 


(2.3) 


81 


(2.1) 


84 


(2.3) 


84 


(1.7) 


Counseling 


83 


(1.6) 


83 


(1.6) 


76 


(1.2) 


76 


(1.9) 


77 


(1.6) 
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Table 7: 

Degree of relevance of competencies in selecting assessment procedures for professional 
development 



Assessment practices 




Degree of importance for 8 






A 1 


c 2 


T 3 


1 . Examine specimen sets, disclosed tests or samples of questions. 


59.5 

(7.9) 


68.5 

(3.6) 


61.3 

(5.3) 


2. Knowledge about instruments and techniques to assess works 
conditions. 


72.4 

(7.3) 


69.2 

(4.1) 


71.8 

(4.2) 


3. Should be skilled in choosing assessment methods appropriate for 
instructional decisions. 


76.7 

(5.9) 


81.5 

(3.2) 


89.5 

(2.7) 


4. Review the performance of test takers of different backgrounds 
when samples of sufficient size are available. 


69.0 

(6.9) 


69.9 

(4.7) 


58.1 

(4.9) 


5. Knowledge about instruments and techniques to assess leisure. 


87.1 

(3.2) 


85.5 

(3.1) 


74.6 

(4.0) 


6. Selection of assessment method-basis. 


87.0 

(4.4) 


85.0 

(2.5) 


81.0 

(3.4) 


7. Ability to evaluate assessment in terms of validity and reliability. 


82.8 

(5.3) 


77.5 

(3.3) 


74.6 

(4.2) 


8. Read the materials provided by test developers and avoid using 
tests for which unclear or incomplete information is provided. 


91.4 

(2.6) 


83.7 

(3.0) 


81.9 

(3.7) 


9. Using norms correctly. 


86.0 

(4.2) 


82.0 

(2.9) 


79.0 

(3.8) 


1 0. Read independent evaluations of a test and of possible alternative 
measures. 


63.8 

(6.6) 


68.1 

(3.9) 


65.3 

(4.5) 


1 1 . Knowledge about instruments and techniques to assess personal 
characteristics. 


85.3 

(4.6) 


88.0 

(2.3) 


80.7 

(3.4) 


12. Define the purpose and then select a test for that purposes and that 
population to be tested. 


70.7 

(7.3) 


76.5 

(3.9) 


78.6 

(4.2) 


13. Evaluate the procedures used by test developers to avoid 
potentially insensitive content or language. 


76.7 

(5.8) 


64.5 

(4.2) 


71.4 

(4.0) 


14. Select and use only those tests for which the skills needed to 
administer the test and interpret scores correctly are available. 


69.0 

(5.8) 


71.7 

(3.9) 


70.2 

(3.4) 


15. Ability to select and use instruments proper to client’s physical. 


70.7 

(6.8) 


80.8 

(3.4) 


73.4 

(3.9) 


1 6. Ability to select assessment techniques appropriate for 
group/individual administration. 


81.9 

(5.7) 


73.9 

(3.7) 


80.2 

(3.2) 


17. Use appropriately modified forms or administration Warn test 
users procedures for test takers with handicapping conditions. 


70.7 

(6.9) 


72.5 

(4.5) 


70.2 

(4.3) 


1 8. Become familiar with how and when the test was developed and 
developed and tried out. 


75.9 

(5.9) 


76.5 

(3.5) 


72.2 

(4.3) 
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19. Knowledge about variables such as ethnicity. 

20. Standard test— meaning of measurement error. 



a On a self-report of importance grade 

1 Percentage of importance for administrator 

2 Percentage of importance for counselor 

3 Percentage of importance for teacher 



77.6 


82.3 


(6.0) 


(3.2) 


72.0 


68.0 


(6.7) 


(4.2) 




78.6 

(4.2) 

75.0 

(3.9) 
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Table lb: 

Degree of relevance of competencies in developing assessment procedures for professional 
development 
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Item description 




Importance grade for a 






A 1 


c 2 


T 3 


1 . Item construction — essay/performance. 


75.0 


61.2 


73.0 


(5.8) 


(4.3) 


(4.1) 


2. Should be skilled in developing assessment methods. 


81.0 


75.0 


83.9 


(5.4) 


(3.7) 


(3.5) 


3 . Investigate potentially useful sources of information, to 


78.4 


70.6 


66.5 


corroborate the information provided by tests. 


(5.6) 


(4.0) 


(3.9) 


4. Made assessment least measurement error. 


75.0 


65.0 


78.0 




(5.9) 


(4.7) 


(3.9) 


5 . Ascertain whether the test content and norm group(s) or 


91.4 


90.6 


86.3 


comparison group(s) are appropriate. 

a ^ 1 i 


(2.2) 


(2.1) 


(3.1) 



On a self-report of importance grade 
1 Percentage of importance for administrator 
! Percentage of importance for counselor 
1 Percentage of importance for teacher 



0 

ERIC 
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Table 7_c: 

Degree of relevance of competencies in administering assessments for professional development 



Item description 




Importance grade for 3 






A 1 


c 2 


T 3 


1 . Should be skilled in administering externally-produced 
assessment methods. 


60.3 

(6.8) 


58.3 

(4.2) 


52.4 

(5.1) 


2. Knowledge of and ability to effectively and appropriately use 
computer-assisted assessment measures and techniques. 


58.6 

(6.5) 


56.2 

(4.8) 


57.3 

(4.7) 


3. The teacher should be skilled in administering teacher-produced 
assessment methods. 


74.1 

(5.9) 


58.7 


75.8 

(3.8) 


4. Ability to administer, score and report findings from career 
assessment instruments. 


73.2 

(7.0) 


72.1 

(4.0) 


68.1 

(4.6) 



a On a self-report of importance grade 

1 Percentage of importance for administrator 

2 Percentage of importance for counselor 

3 Percentage of importance for teacher 
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Table 7_d: 

Degree of relevance of competencies in scoring assessments for professional development 



Item description 




Importance grade for a 






A 1 


c 2 


T 3 


1 . Should be skilled in scoring externally-produced tests. 


58.6 


49.3 


43.5 


(6.8) 


(3.9) 


(4.9) 


2. Should be skilled in scoring teacher-produced tests. 


79.3 


62.7 


75.0 


(5.6) 


(4.4) 


(3.6) 


3. Should be able to explain the rationale for choosing grading 


73.3 


79.3 


86.3 


practices. 


(6.8) 


(3.6) 


(3.2) 


4. Recognize sound grading practice 


82.0 


71.0 


74.0 


(5.7) 


(3.8) 


(4.1) 


5. Reliability of tests for grading. 


88.0 


80.0 


87.0 


(4.0) 


(3.1) 


(2.4) 


6. Explain basis for grade. 


83.0 


70.0 


78.0 


(5.1) 


(3.7) 


(3.3) 


7. Weighting test scores to give grades. 


70.0 


61.0 


66.0 


(5.9) 


(4.0) 


(4.7) 



a On a self-report of importance grade 
5 Percentage of importance for administrator 

2 Percentage of importance for counselor 

3 Percentage of importance for teacher 
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Table 7_e: 

Degree of relevance of competencies in interpreting assessment results for professional 
development 



Item description 




Importance grade for 3 






A 1 


c 2 


T 3 


1 . Know basis for comparing schools' test scores. 


59.0 


50.0 


59.0 




(7.9) 


(4.5) 


(4.5) 


2. Explain how any passing scores were set and gather evidence to 


77.6 


73.9 


74.2 


support the appropriateness of the scores. 


(6.0) 


(3.8) 


(3.4) 


3. Ability to interpret data from assessment instruments and present 


75.0 


75.7 


73.0 


the results to client and to others designated by client. 


(5.7) 


(3.6) 


(3.6) 


4. Test as only criterion for grade. 


75.0 


77.0 


77.0 




(6.4) 


(3.8) 


(4.2) 


5 . Standard test data most useful for classroom 


70.0 


60.0 


66.0 




(5.6) 


(3.6) 


(4.1) 


6. Should be able into consideration students characteristics as part 


81.9 


73.9 


73.8 


of score interpretation. 


(3.7) 


(3.5) 


(3.6) 


7. Should be skilled in using assessment results when making 


81.9 


72.5 


81.8 


decisions. 


(4.8) 


(3.6) 


(2.7) 


8. Interpret teacher-made test score. 


78.4 


63.0 


77.0 




(5.5) 


(4.7) 


(3.5) 


9. Obtain evidence to help show that the test is meeting its intended 


84.5 


72.5 


84.7 


purpose(s). 


(5.7) 


(3.8) 


(2.5) 


10. Should be able to establish validity of decisions made from 


79.3 


76.8 


79.4 


assessment results. 


(5.6) 


(3.4) 


(3.5) 


1 1 . Avoid using tests for purposes not specifically recommended by 


73.3 


80.1 


66.5 


the test developer. 


(5.9) 


(3.0) 


(4.8) 


12. Interpret percentile band scores. 


64.0 


57.0 


50.0 




(6.9) 


(4.4) 


(5.2) 


13. Obtain information about the scale used for reporting scores, the 


77.6 


71.4 


72.2 


characteristics of any norms or comparison group(s). 


(5.3) 


(3.9) 


(3.8) 


14. Using tests for resource allocation. 


73.3 


59.0 


60.0 




(5.5) 


(4.1) 


(4.3) 


15. Interpret Grade Equivalency score. 


55.0 


47.0 


48.0 




(7.6) 


(4.5) 


(5.0) 


1 6. Should be skilled in interpreting the results of externally produced 


73.3 


62.3 


58.1 


assessment methods. 


(5.1) 


(4.1) 


(4.4) 


17. Should be skilled in interpreting the results of teacher-produced 


75.9 


59.4 


77.8 


assessment methods. 


(5.7) 


(4.4) 


(3.7) 



a On a self-report of importance grade 

1 Percentage of importance for administrator 

2 Percentage of importance for counselor 
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3 Percentage of importance for teacher 
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Table 7_f: 

Degree of relevance of competencies in comunicating assessment results for professional 
development 



Item description 




Importance grade for 3 






A 1 


c 2 


T 3 


1 . Teachers should be skilled in communicating assessment results 


83.6 


76.8 


88.3 


to students. 


(6.1) 


(3.5) 


(3.0) 


2. Ability to write a thorough and substantiated report of assessment 


83.6 


78.3 


80.6 


results. 


(4.5) 


(3.1) 


(3.7) 


3. Teachers should be skilled in communicating assessment results 


71.5 


69.2 


79.8 


to parents. 


(6.2) 


(3.7) 


(3.3) 


4. Teachers should be skilled in communicating assessment results 


67.2 


57.6 


65.7 


to others educators. 


(6.0) 


(3.7) 


(3.6) 


5 . Explaining discrepancy between classroom and standard test 


66.0 


57.0 


60.0 


scores. 


(7.7) 


(4.4) 


(4.6) 


6. Ability to assist client and others designated by the client to 


75.0 


61.6 


63.7 


interpret data from assessment instruments. 


(5.7) 


(4.7) 


(4.3) 



a On a self-report of importance grade 

1 Percentage of importance for administrator 

2 Percentage of importance for counselor 

3 Percentage of importance for teacher 
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Table 7 _g: 

Degree of relevance of competencies in ethical/legal issues for professional development 



Item description 




Importance grade for a 






A 1 


c 2 


T 3 


1 . Knowledge of state and federal statutes relating to client 


57.8 


57.2 


46.4 


confidentiality. 


(8.2) 


(5.1) 


(5.3) 


2. Tell test takers or their parents/guardians how long scores will be 


74.1 


64.9 


66.9 


kept on file. 


(5.7) 


(4.0) 


(3.9) 


3. Ability to apply ethical standards to career counseling and 


78.4 


74.6 


68.9 


consulting situations, issues, and practices. 


(6.3) 


(3.7) 


(5.2) 


4. Knowledge acceptable actions on standardized tests. 


84.0 


86.0 


75.0 




(4.9) 


(2.7) 


(4.2) 


5 . Describe the procedures that test takers or their parents/guardians 


57.8 


50.4 


51.6 


may use to register complaints and have problems resolved. 


(7.6) 


(4.8) 


(4.8) 


6. Display of grade-privacy. 


75.0 


83.0 


78.2 




(6.1) 


(3.1) 


(3.7) 


7. Ability to recognize situations involving interpretation of ethical 


69.0 


70.0 


66.9 


standards. 


(6.3) 


(4.0) 


(4.3) 


8. Knowledge about ethical issues related to career counseling with 


71.5 


73.2 


64.5 


women, cultural minorities, immigrants, the disabled, the elderly, 


(6.9) 


(3.8) 


(5.0) 


and persons with the AIDS virus. 








9. Should be skilled in recognizing unethical, illegal, and otherwise 


78.4 


70.3 


73.4 


inappropriate assessment methods. 


(5.5) 


(4.1) 


(4.1) 


10. Knowledge about current ethical and legal issues which affect the 


75.0 


70.0 


62.1 


practice of career counseling. 


(5.5) 


(4.2) 


(4.9) 


1 1 . Knowledge about current ethical/legal issues with regard to the use 


63.8 


60.5 


50.8 


of computer-assisted career guidance. 


(7.1) 


(4.5) 


(4.8) 


12. Provide test takers the information they need to be familiar with the 


77.6 


76.4 


72.2 


coverage of the test. 


(5.3) 


(3.4) 


(4.3) 


13. Knowledge about the code of ethical standards of relevant 


73.3 


68.5 


68.9 


professional organizations. 


(6.3) 


(4.0) 


(4.6) 


14. Provide test takers or their parents/guardians with information about 


72.4 


68.8 


71.8 


rights test takers. 


(6.4) 


(3.8) 


(3.7) 


15. Provide test takers or their parents/guardians with information to 


75.0 


72.1 


67.3 


help them judge whether the test should be taken. 


(5.8) 


(3.9) 


(4.2) 



a On a self-report of importance grade 

1 Percentage of importance for administrator 

2 Percentage of importance for counselor 

3 Percentage of importance for teacher 
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Table 8: 



Comparative results on assessment competencies from two international studies 



Item description 


Impara & Plake (1995) a 


Cab, Arce & Cisneros 
(2001) b 




A 1 


C 2 


T 3 


A 4 


C 5 




Standard 1: Choosing assessment methods 














Selection of assessment method — basis 


.99 


.99 


.99 


.87 


.85 


.81 


Standard test-meaning of measurement error 


.79 


.91 


.76 


.72 


.68 


.75 


Using norms correctly 


.76 


.85 


.77 


.86 


.82 


.79 


Standard 2: Developing assessment methods 














Teacher made assessment least measurement error 


.68 


.75 


.67 


.75 


.65 


.78 


Determining validity 


.91 


.90 


.94 


.83 


.78 


.75 


Item construction-essay/performance 


.82 


.83 


.78 


.75 


.61 


.73 


Standard 3: Interpreting assessment results 














Interpret teacher-made test score 


.82 


.83 


.78 


.78 


.63 


.77 


Interpret Grade Equivalency score 


.77 


.79 


.63 


.55 


.47 


.48 


Interpret percentile band scores 


.80 


.78 


.60 


.64 


.57 


.50 


Standard 4: Using assessment results in decision making 














' Standard test data most useful for classroom 


.55 


.51 


.52 


.70 


.60 


.66 


Basis for comparing schools’ test scores 


.63 


.59 


.52 


.59 


.50 


.59 


Explaining discrepancy between classroom and standard 
test scores 


.55 


.57 


.47 


.66 


.57 


.60 


Standard 5: Using assessment results in grading 














Weighting test scores to give grades 


.38 


.36 


.34 


.70 


.61 


.66 


Reliability of tests for grading 


.21 


.24 


.15 


.88 


.80 


.87 


Recognize sound grading practice 


.85 


.85 


.85 


.82 


.71 


.74 


Standard 6: Communicating assessment results 














» Explain basis for grade 


.98 


.99 


.99 


.83 


.70 


.78 


Interpret stanine 


.59 


.67 


.37 


* 


* 


* 


• Using tests for resource allocation 


.91 


.93 


.92 


.73 


.59 


.60 


Standard 7: Recognizing unethical assessment practices 














1 Display of grade— privacy 


.98 


.96 


.98 


.75 


.83 


.78 


• Test as only criterion for grade 


.91 


.91 


.87 


.75 


.77 


.77 



O 




29 



30 



Acceptable actions on standardized tests 



.90 .95 .94 .84 .86 .75 



* Not included for being culturally irrelevant. 

a On a test of measurement concepts 

1 Proportion correct administrator 

2 Proportion correct counselor 

3 Proportion correct teacher 



b On a self-report of importance grade 

4 Proportion of importance for administrator 

5 Proportion of importance for counselor 

6 Proportion of importance for teacher 
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